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An increased emphasis on writing standards has led many U.S. states to incorporate on-demand 
writing assessments into their test-based accountability system. We argue this creates political and 
pedagogical tensions for teachers to navigate. We discuss how rubric conceptualization (1) is a 
process wherein a teacher iteratively (co-)constructs meaning from a rubric’s design via classroom 
instruction; (2) is informed by implicit theories of learning; and (3) often requires a teacher to 
negotiate the competing pedagogical and political meanings of a rubric. While test-based account- 
ability frameworks promote rubric use that equates learning with student achievement, rubric 
conceptualization is a process where teachers have some agency to resist behaviorist approaches 


to instruction. 


[" writing instruction, rubrics can be a contentious topic. Proponents of 
rubrics often talk about their utility in a variety of ways. For one, writing 
rubrics can provide explicit expectations to students, which can facilitate 
feedback and self-assessment (Andrade, 2006; Jonsson & Svingby, 2007; Sad- 
dler & Andrade, 2004). Researchers have also claimed that writing rubrics 
promotes more reliable scoring and provides a means to track students’ writ- 
ing development over time (Brookhart & Chen, 2015; Correnti, Matsumura, 
Hamilton, & Wang, 2013). Rubric critics, on the other hand, often lament 
an undue focus on efficiency, how rubrics negate the necessarily subjective 
nature of writing, and how they narrow the complex task of writing into 
performance standards that misrepresent underlying values and purposes 
of writing (Broad, 2003; Kohn, 2006; Wilson, 2007). Historically, these per- 
spectives have meant that discussions often amount to taking a pro- versus 
anti-rubric stance rather than articulating under what conditions rubrics 
can be used appropriately (Turley & Gallagher, 2008). 
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We seek to reframe the discussion around writing rubrics by show- 
ing how rubrics can both illuminate and help educators negotiate various 
tensions that occur in the processes of writing instruction and assessment. 
In particular, we focus on state writing rubrics and their dueling identi- 
ties as a high-stakes policy tool and a formative classroom assessment tool. 
As a high-stakes policy tool, rubrics are typically valued for their ability to 
measure writing skills in ways that are “valid,” “reliable,” and “objective.” 
With the rise of test-based accountability systems in the past 50 years and the 
more recent adoption of the Common Core State Standards (CCSS), writing 
rubrics function more and more like policy tools for accountability rather 
than assessment tools that support learning (Welner, 2013). The use of rubrics 
as a policy tool for accountability is also part of a much longer tradition of 
embedding behaviorist learning theory (Shepard, 2000) and principles of 
scientific management (Au, 2011) into standardized testing, which began in 
the early twentieth century. Critiques of these test-based accountability sys- 
tems abound, and for good reason. High-stakes assessments have been shown 
to lead to a wide variety of negative consequences, including curriculum 
narrowing (Au, 2007; Jennings & Bearak, 2014), teacher-centered instruc- 
tion (Au, 2007), the neglect of democratic aims of education (Ben-Porath, 
2013); the pushout of lower performing high school students (Tuck, 2012), 
and the exacerbation of racial inequality in education (Au, 2016; Horsford, 
2017), just to name a few. 

Promoting sociocultural theories of learning has been one response 
literacy researchers and teacher educators have made to address these 
inequalities and inadequacies. Sociocultural approaches to instruction and 
assessment focus on meaning making and view learning as a dynamic set 
of interactions among people, artifacts, and settings that occur across time 
(Lee, 2008). This contextualized view of literacy and writing seeks to create 
authentic rhetorical contexts (Newell, Beach, Smith, & VanDerHeide, 2011) 
and stands in opposition to the “objective” achievement-based view of learn- 
ing embedded within behaviorism and standardized assessments. 

Behaviorism and sociocultural theory’s views and assumptions about 
learning and literacy create tensions in our public education system that 
teachers must (sub)consciously navigate. Given the pervasive reality of 
test-based accountability systems in U.S. public schools and the behaviorist 
assumptions that undergird them, teachers must actively negotiate these 
tensions and constraints to engage in socially and culturally responsive 
forms of instruction (Ladson-Billings, 2014). We focus on writing rubrics in 
this article to provide a concrete instantiation of the complex negotiation 
processes teachers engage in when making instructional decisions. We posit 
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that the process of rubric conceptualization can help teachers negotiate these 
tensions in rubric use and writing instruction more broadly. As a result, we 
view engagement with rubric conceptualization as a rich space for inquiry 
and praxis among teachers, teacher educators, researchers, and community 
members. While we believe only broad structural changes to the educational 
system will be able to lessen these tensions, we think change can, and should, 
occur. We position teachers, teacher educators, students, and community 
members as agents who can, despite tensions and constraints, exert agency 
to buffer, resist, and reappropriate structures in ways that value and develop 
students’ social, cultural, political, and experiential knowledge. 


What Is Rubric Conceptualization? 


We identify three distinct but interrelated processes in rubric use: design, 
conceptualization, and implementation. In defining and distinguishing each 
concept, we seek to disambiguate these processes. While state rubrics often 
mandate the genre and desired writing skills within the rubric design, we 
argue that teachers can exercise some agency in how they conceptualize 
and implement those rubrics with their students. 

Rubric design deals with concrete features of the rubric, such as its 
content and layout. Prior research has examined the benefits and drawbacks 
of different rubric designs. For example, in terms of rubric layout, research 
has compared the effectiveness of holistic rubrics, which use a list of crite- 
ria to give one overall score for a student’s piece of writing, and analytic 
rubrics, which give multiple scores to represent the level of performance 
on several dimensions such as evidence use and organization (Smit & Birri, 
2014). The general consensus is that while holistic rubrics are reliable and 
probably more efficient in that they take less time to use, analytic rubric 
designs present more precise and useful data for understanding student 
performance and growth, which may be especially useful for the purpose 
of formative assessment (Jonsson & Svingby, 2007; Reznitskaya, Kuo, Glina, 
& Anderson, 2009). 

Rubric implementation is how teachers use the rubric to support 
teaching and learning within the classroom. This includes how rubrics are 
embedded in the curriculum, how rubrics are framed as part of the writ- 
ing process, how and when students interact with the rubric, how teachers 
interact with students about rubric content to aid the learning of concepts 
within a particular genre such as argument writing, and how all of these 
interactions develop iteratively over time across the school year. While 
rubric implementation can be viewed as merely technical or a question 
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of fidelity (e.g., Andrade & Boulay, 2003; Andrade, Du, & Wang, 2008), we 
view implementation as a practice that is informed by practical experience, 
relationships, and knowledge of the community. This view of rubric imple- 
mentation is relatively understudied since most research studies have been 
interventions that seek to understand the effect of rubric use on student 
outcomes. In this work, researchers have largely used experimental and 
quasi-experimental designs that aim for a standardized implementation 
(i.e., fidelity) to reduce confounding factors (see Jonnson & Svingby, 2007, 
and Brookhart & Chen, 2015, for meta-analyses of these types of studies). 
As a result, these designs do not lend themselves to studying how and why 
teachers implement rubrics in different ways. 

We pose rubric conceptualization as a mediating step within and be- 
tween the processes of design and implementation. Conceptualization entails 
asking questions such as, How can this rubric support teacher instruction? 
Student learning? More technically, we define rubric conceptualization as 
the context-specific process where a stakeholder conceives of the possible 
function(s) that a rubric can serve in assessment, instruction, and learning. 
Table 1 shows how this process can take different forms among different 
stakeholders. For teachers, the process of rubric conceptualization includes 
determining if and how the rubric can support student learning around 
writing concepts, which instructional goals the rubric aligns with, and what 
types of instructional activities the rubric is (not) useful for. We view rubric 
conceptualization as a potentially iterative and reciprocal process where 
teachers and students can co-construct meaning through their interactions 
throughout the school year. 

Other stakeholders also engage in rubric conceptualization, albeit 
in ways more indirectly related to classroom instruction and learning. For 
example, when an instructional designer or policymaker selects rubric con- 
tent that aligns with state standards, they are not only prioritizing particular 
writing genres and concepts but also (implicitly) endorsing certain functions 
and purposes for rubrics. These functions may be focused on summative or 
accountability-based forms of assessment, which can limit the rubric’s use 
as a formative assessment tool. The mandated use of rubrics can also lead 
to mandated curricula, which both result in reifying conceptualizations of 
writing and literacy that are aligned with test-based accountability systems 
(we discuss an example in “What Could Sociocultural Rubric Conceptualiza- 
tion Look Like?” later in this article). 

While higher-level stakeholders may confound rubric design with con- 
ceptualization and/or implementation, these three processes are important 
to distinguish because the meaning and use of a rubric is neither static nor 
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Table 1. Stakeholders’ Roles in Rubric Creation and Use 


Process Process Potential Description of Stakeholders’ Roles 
Description Stakeholders 

Design What is included and | Policymakers In states that have high-stakes writing assess- 
emphasized inthe | Test makers ments, the design of writing rubrics for ELA 
rubric content and often shifts to state policymakers and testing 
layout? Administrators | corporations. 


Administrators? 


Some ELA teachers may be able to adapt these 
rubrics (e.g., make them more “student friend- 


should this rubric 
support teaching 
and learning? 


(Policymakers; 
Test makers) 


Teachers ly”). States without accountability testing would 
be more likely to have teachers who choose or 
design their own writing rubrics. 

Conceptualization | How can and how Designers Whoever designs the writing rubric is also (im- 


plicitly) engaging in rubric conceptualization: 
they have an intended goal and/or function of 
the rubric in mind. 


Implementers: 
Teachers 


When teachers plan to implement the rubric, 
they are conceptualizing what learning goals 
they think can be achieved and what instruc- 
tional tasks they are useful for. These rubric 
functions may or may not align with those of the 
rubric designers. 


Students 


Students conceptualize what a writing rubric 
is useful for based on their identity, their prior 
experiences as a writer, and how teachers 
frame and implement rubrics during classroom 
instruction. 


Implementation 


How is the rubric 
used to support 
teaching and learn- 
ing? 


Mandated 
curricula (state 
or district poli- 
cymakers) 


Mandated curricula and lessons that dictate 
the content of lessons (texts, writing prompts, 
and other instructional activities) can constrain 
possible classroom implementations. 


Teachers 


However, teachers have (some) agency in deter- 
mining what specific instructional strategies, 
pedagogical moves, and instructional tools to 
use during instruction. 


predetermined by its design. Instead, a rubric design is the raw material 


from which teachers construct their own context-specific meanings. This 


is not to say that the raw materials (i.e., rubric designs) do not matter but 


that only considering the rubric design is insufficient for understanding the 


iterative processes of rubric use.! 


In addition, we view rubric conceptualization as not only a cognitive 


act but, like implementation, also informed by experience, relationships, 


and knowledge of the local context. We see conceptualization as a process 


akin to translation; teachers and administrators must take a rubric created 


in a separate context and figure out what meaning it can have for (1) their 


own writing instruction and, (2) their students’ learning. We have chosen 
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to focus this article on rubric conceptualization because this mediating 
process is typically overlooked and because we think it has potential to help 
educators navigate tensions between sociocultural forms of instruction and 
the pedagogical and political realities imposed by today’s accountability as- 
sessments and the systems built around them. 


Interpretive Flexibility and Three Different Functions of Rubric 
Use 


With rubric design, conceptualization, and implementation disambiguated, 
we now shift to the possibilities and consequences of rubric conceptualiza- 
tion. We posit that the more teachers are oriented toward sociocultural 
instruction, the more they will experience tensions in the conceptualiza- 
tion and implementation of state writing rubrics. 
A key aspect of state rubrics is To support this claim, we first explain how rubrics 
that they are both policy tools are boundary objects and, as a result, how rubric 
and pedagogical tools. conceptualization is a form of interpretive flexibility. 
Next, we outline how this interpretive flexibility leads 
to three possible rubric functions. We discuss how these functions can be 
compatible with and/or in contention with behaviorist and sociocultural 
learning theories. How these tensions are (not) perceived and navigated is 
consequential to teacher practice and student learning, which we further 
discuss in the next section. Ultimately, we view rubric conceptualization as 
capable of helping to negotiate, but not fully resolve, these ideological ten- 
sions because sociocultural forms of instruction are generally not aligned 

with the values and logics of test-based accountability systems. 


Rubrics as Boundary Objects 


A key aspect of state rubrics is that they are both policy tools and pedagogical 
tools. State rubrics are policy tools in the sense that they are part of large-scale 
externally mandated state writing tests that are used to reward or sanction 
students, teachers, and/or schools based on their performance (Au, 2009; 
Hamilton, 2003). At the same time, rubrics can also be pedagogical tools 
when they are used to help teachers and students develop understandings 
and skills related to writing. These dual identities and the fact that differ- 
ent stakeholders within and between different levels of the educational 
bureaucracy can conceptualize state rubrics in different ways (as seen in 
Table 1) means that state rubrics have interpretive flexibility. More techni- 
cally, we propose that because state writing rubrics traverse all levels of the 
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educational bureaucracy, they are boundary objects that can have different 
meanings in different contexts and work processes. 

A boundary object has “different meanings in different social worlds 
but their structure is common enough to more than one world to make them 
recognizable, a means of translation” (Star & Griesemer, 1989, p. 393). The 
fact that state rubrics can take on different meanings in different contexts 
based on their use and interpretation (i.e., rubric conceptualization and 
implementation) means that state rubrics have interpretive flexibility. As a 
result, teachers can vary with how and to what extent they engage with state 
rubrics as a policy tool and/or a pedagogical tool. In particular, we posit that 
there are three key functions that state rubrics can serve depending on how 
they are conceptualized and implemented: assessment for pedagogy, assess- 
ment for accountability, and assessment for measurement. While teachers often 
implicitly engage these varying functions, how they are conceptualized and 
implemented has consequences for teaching and learning. 


Three Rubric Functions and Their (In)compatibility 


Using a writing rubric as an assessment tool for pedagogy entails that the 
rubric is helping to iteratively and reciprocally build understanding between 
and among teachers and students about key skills, processes, and reasons 
for writing in a particular genre. Assessment for pedagogy is in alignment 
with sociocultural theories of learning because it views writing as a social 
process that is embedded within a particular context; that the individual 
and their context interact in interdependent ways (Gresalfi, 2009). This 
means that a teacher does not think that the language and words within the 
rubric have an obvious or predetermined meaning. Instead, a major focus 
of writing instruction is the co-construction of meaning around writing 
concepts. Discussions are used to help students iteratively develop meaning 
around key writing concepts in the rubric (e.g., “evidence” and “claim” ) so 
that students develop understanding around both how and why they should 
use these concepts in their writing. Even further, these writing tasks and 
discussions would ideally be connected to authentic writing tasks that are 
directed toward issues and audiences that are relevant to the students’ lives 
and communities (Kinloch, 2010). In this way, rubric use becomes less about 
evaluating student work and more about coming to understand the meaning 
and purpose of writing in different contexts. 

The second function, assessment for accountability, entails that writing 
rubrics are conceptualized and implemented as a high-stakes summative as- 
sessment tool that evaluates the student, teacher, and/or school. Typically, 
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practitioners receive messages about assessment for accountability through 
the rubric’s use as a policy tool. While high-stakes reading and math tests 
have traditionally been the focus of state accountability systems since No 
Child Left Behind was enacted in 2001, CCSS has increased the number of 
states with standardized writing assessments, especially text-based writing 
tasks. A 2016 study found that 46 of 50 states had an on-demand essay on 
their standardized exams, with 39 having some form of text-dependent essays 
(Behizadeh & Pang, 2016, p. 32). As states adopt and prioritize standardized 
tests in writing and develop a subsequent focus on aligning writing curricula 
and instruction to the assessments, the writing rubrics’ political focus on 
performance and accountability can displace its formative, pedagogical focus 
on learning and meaning making. This is because test-based accountability 
systems typically synonymize student achievement scores with student 
learning (and teacher effectiveness) despite the low consequential valid- 
ity of the tests (Behizadeh, 2014; Welner, 2013). Equating “learning” with 
performance is compatible with behaviorist theories of learning because 
learning and knowing are viewed as the accumulation of stimulus-response 
associations, routinized practice and tightly sequenced learning objectives 
(Greeno, Collins, & Resnick, 1997). Relatedly, assessment for accountability 
also promotes content control, pedagogic control, and bureaucratic control 
(Au, 2009). Mabry (1999) describes how this looks in regards to rubrics: 


Rubrics are designed to function as scoring guidelines, but they also 
serve as arbiters of quality and agents of control. Moreover, the control 
is not limited to assessment episodes but influences curriculum choices, 
restricts pedagogical repertoires, and restrains student expression and 
understanding. (p. 674) 


In the next section, we discuss one state scenario that exemplifies this shift 
toward assessment for accountability. 

Last, assessment for measurement means that rubrics are viewed and/ 
or used as a means to measure student performance or the development of 
students’ skills and understandings. Whether assessment for measurement 
is compatible with the other two functions depends on how measurement 
is defined and operationalized (as outlined in Figure 2). We suspect that 
many may immediately associate assessment for measurement as being 
aligned with the accountability function of rubrics, given the emphasis on 
cut scores, proficiency levels, and “data-driven instruction” in test-based ac- 
countability systems (Bennett, 2011; Shepard, 2015; Welner, 2013). These uses 
of measurement are indeed compatible with the accountability function of 
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rubrics because they utilize quantification to make a high-stakes summative 
evaluation of student writing. 

However, assessment for measurement is not necessarily at odds with 
the pedagogical function of rubrics. A form of measurement that is compat- 
ible with assessment for pedagogy is one that does not use numbers as the 
end itself, but as a means to a larger goal: describing and helping improve stu- 
dents’ understanding and skills over time. When used this way, measurement 
does not emphasize the number score on a rubric (e.g., a 2 in evidence use) 
but what the score means in terms of skills and concepts within a progression 
of learning (Shepard, 2013). While this may sound like a subtle distinction, 
it can significantly alter a classroom’s cultural assessment practices, which 
are “the social norms and meanings associated with assessment processes 
in classrooms” (Shepard, Penuel, & Pellegrino, 2018, p. 28). 

When measurement is framed as an end in itself, it likely promotes 
a “performance orientation” among students. This focus on performance 
creates an external form of motivation in line with behaviorism. While 
the performance orientation is, in general, the status quo in schools, we 
argue that different conceptualizations of rubrics and assessment cultural 
practices could support a “learning orientation” instead. Such practices 
frame measurement as a means to observe and develop understanding. This 
orientation supports both internal (sociocultural) motivational structures 
for improvement and promotes the idea that writing skills are developed 


iteratively over time. 


Tensions within and between Rubric Conceptualizations and 
Functions 


Different theories of learning and different assessment practices can lead 
to different conceptualizations of rubric functions, as shown in Figure 1. 
Practically, these different rubric functions would lead to different learning 
environments for students. For example, a context that is compatible with 
assessment for accountability (bottom left of Figure 1) is when teachers and 
students focus on the rubric scores (i.e., performance) and engage in minimal 
substantive meaning making around what those scores mean in terms of 
learning and writing. In such a case, students receive their scores, but are 
likely provided minimal feedback as to why they received that score. While 
students can determine whether they received a “good/high” or “bad/low” 
score, they are left to make their own inferences about what they learned and 
why they scored that way. This use of rubrics mirrors high-stakes account- 
ability testing, where teachers and students typically receive individual and 
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aggregated rubric scores but no personalized feedback. This accountability 
function (implicitly) operationalizes a behaviorist learning theory because 
it assumes students can learn how to write from a rubric telling them what 
the standards and expectations are. In contrast, if rubric scores are viewed 
as a means to iteratively discuss how student skills and understandings are 
developing over time, then the rubric is serving an assessment for pedagogy 
function (top right in Figure 1). 

Research shows that teachers can hold multiple beliefs about assess- 
ment simultaneously (Barnes, Fives, & Dacey, 2017; Harris & Brown, 2009; 
Remesal, 2011). Given that rubrics are part of assessment practices, we 
posit that teachers can also have multiple conceptualizations of rubrics. 
This means that teachers’ rubric use (and other aspects of their writing in- 
struction) may move around different areas of Figure 1. Tensions can occur 
when multiple functions and their underlying learning theories come into 
conflict. For instance, teachers could blend the assessment for pedagogy and 
assessment for accountability functions (i.e., the light gray circles in Figure 
1), either by being oriented toward a sociocultural view of learning but with 
an explicit focus on performance or, conversely, have a focus on learning 
but a behaviorist learning theory. The closer to the upper right quartile that 


Learning as a complex 
social, cultural, cognitive, 
and political endeavor 


Sociocultural learning 
theory, but focus on 
performance 


What is learning? 


Learning as transmission 
and acquisition 
+) sSsSsSs<C—F—F 
Performance Learning 
What is the focus of instruction and assessment? 


Figure 1. Relationship between Learning Theories and Rubric Functions 
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a teacher’s operationalization of rubrics lie, the more aligned it is with so- 
ciocultural learning theory, while the lower left represents alignment with 
behaviorism. The sources of these tensions and multiple conceptualizations 
are likely complex, though one possible source is the macro-level influence 
of test based accountability systems, which are more likely to promote 
behaviorist-aligned pedagogies (Au, 2007; Jennings & Bearak, 2014). At the 
same time, we also view contextual factors at the classroom (micro) and 
school (meso) levels as creating particular opportunities and constraints 
that teachers must navigate. 

The potential for tensions in a teacher’s rubric use further increases 
once we consider the processes of rubric design and rubric implementation. 
This is because misalignments can exist within and between the three rubric 
processes (design, conceptualization, and implementation), as shown in 
Figure 2. For instance, a sociocultural conceptualization of the state rubric 
could lead teachers to feel their instruction is misaligned with the account- 
ability function of the state test. Teachers’ implementation of the rubric 
could respond to such tensions in different ways, as the different ovals in 
Figure 1 show. Teachers could defer from state accountability policies (bot- 
tom left), resist these pressures to adopt an assessment for pedagogy function 
(upper right), or enact a hybrid function. Within this negotiation process, 
we hypothesize that rubric conceptualization plays a significant mediating 
role, as shown by the thickness of the arrows in Figure 2. While we anticipate 
that negotiations in rubric conceptualization and functions are not always 
conscious, we believe that making conceptualization an explicit topic of 
teacher education and professional learning communities (PLCs) could 
support teachers in using rubrics in ways that are aligned with assessment 


for pedagogy and sociocultural learning theory more broadly. 


The Social and Practical Consequences of Rubric Use 


The routines of standardized, summative assessments are often so ingrained 
that we don’t think about how this form of assessment is aligned with the as- 
sumptions of behaviorist and cognitivist theories of learning. These theories 
assume that students must be tested as individuals because social interaction 
is viewed as an impediment to accurate measurement. In addition, “good 
measurement” is viewed as requiring standardized texts and prompts. How- 
ever, such standardization in direct writing assessments (DWAs) deny the 
importance of the local context (Behizadeh, 2014). In addition, DWAs allow 
researchers, practitioners, and policymakers to make limited generaliza- 
tions about students’ overall writing abilities because the target domain of 
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Rubric 
Conceptualization 


How can this 
rubric support 
teaching and 
learning? 


Rubric 


Rubric Design Implementation 


What is 
emphasized in the 
rubric content and 

layout? 


How is the rubric 
used to support 
teaching and 

oe alae mie ww mnie in nie learning? 


Implicit learning 
Intended Rubric Enacted Rubric <----p theory could be 
Function Function aligned or misaligned 
What purpose(s) is the What purpose(s) does the 
rubric expected to serve in rubric serve in instruction ——____-» Contributes to the 
instruction and learning? and learning? rubric function 


Figure 2. Relationships among the Three Processes of Rubric Use and Rubric Functions 


on-demand writing is quite narrow (Kane, Crooks, & Cohen, 1999) despite 
their political use as a measure of student learning, teacher quality, and/ 
or school success.’ 

Critiques of these test-based accountability systems abound, and for 
good reason. High-stakes assessments have been shown to lead to a wide 
variety of negative consequences, including curriculum narrowing (Au, 
2007; Jennings & Bearak, 2014:), teacher-centered instruction (Au, 2007), the 
neglect of democratic aims of education (Ben-Porath, 2013), the pushout of 
lower performing high school students (Tuck, 2012), and the exacerbation 
of racial inequality in education (Au, 2016; Horsford, 2017). This means that 
DWAs are in direct conflict with sociocultural approaches of current research 
where writing—as a component of literacy—requires the consideration of 
students’ sociocultural backgrounds and the creation of authentic rhetorical 
contexts (Newell et al., 2011). 

The problematic behaviorist assumptions and (unintended) conse- 
quences of DWAs have led other researchers to propose new systems of assess- 
ment that better align policy, assessments, and instruction to sociocultural 
theories of learning (Baird, Andrich, Hopfenbeck, & Stobart, 2017; Behizadeh, 
2014; Shepard et al., 2018). For example, Behizadeh (2014) outlines a new 
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vision of large-scale portfolio assessments that promotes of authentic writ- 
ing tasks written for multiple purposes in varied dialects and languages. 
Shepard and colleagues (2018) also propose a new assessment system, one 
where stakeholders within and between different levels of the educational 
system “hold compatible and synergistic visions of learning goals and the 
means to achieve them” (p. 22). 

While we agree that our educational systems should be aligned with 
sociocultural learning theory, we differ in how such a goal may be feasibly 
achieved. Unlike the researchers just mentioned, we do not posit wide- 
ranging policy changes to address this misalignment. Instead, due to the 
historical persistence of behaviorist learning theories and scientific manage- 
ment practices from the early 1900s (Au, 2011; Shepard, 2000), we assume 
educational policies will continue to promote these forms of learning and 
management for the foreseeable future, even despite their well-documented 
history of justifying and reproducing discrimination (Au, 2016). As a result, 
we are interested in using rubric conceptualization to ameliorate the ten- 
sions that teachers experience when they seek to implement sociocultural 
pedagogies in a political climate that typically places too little value on 
these approaches. We believe such tensions are worth negotiating, despite 
the practical and theoretical difficulties, because sociocultural theories of 
learning present opportunities to provide culturally sustaining pedagogies to 
students (Paris, 2012). It also challenges the mainstream rhetoric and goals 
of “closing the achievement gap” that has served to narrow, standardize, 
and decontextualize curricula, especially for students of color (Anderson & 
Cohen, 2015; Tuck & Gorlewski, 2016). 


What Could Sociocultural Rubric Conceptualization Look Like? 


In this section, we contrast two school contexts to show how different con- 
straints can present different opportunities for educators to engage in socio- 
cultural instruction around rubric conceptualization and implementation. 
While the tensions and misalignments among learning theories, curricula, 
classroom instruction, and assessments are often implicit (Baird et al., 2017; 
Shepard et al., 2018), we examine the processes of rubric conceptualiza- 
tion and rubric implementation to make them explicit. We argue that how 
teachers conceptualize and implement rubrics could serve as a lynchpin 
to help teachers align their writing instruction with sociocultural theories 
of learning. 

The two school-based scenarios we discuss sit on opposing sides of a 
constraints spectrum. This spectrum, diagrammed in Figure 3 below, de- 
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scribes the level and intensity of the micro- and meso-level conditions that 

either support or hinder the use of sociocultural literacy practices. Both 
scenarios are derived from the authors’ experiences. The first scenario 

is based on the state context we recently encountered in Louisiana while 
working with fifth- and sixth-grade ELA teachers on an automated essay 

scoring study. While it was not the intended focus 

Opportunities for and constraints of the study, we were struck by how aligned the 
to creating sociocultural learning state, districts, schools, and teachers were in their 
environments for students are — focus and conceptualization of text-based writing 
complexly influenced by meso- assignments. Text-based writing is an emphasized 
and micro-level factors. section of the state ELA exam and performance 

on that state test is used to reward or sanction 

schools and teachers (both politically and monetarily). In particular, emily 

was struck by how different this context for writing instruction was from 

her experiences teaching in Consortium High School (CHS),‘ a New York 

City public school. A composite of emily’s four years of experience at CHS 

serves as the second contrasting scenario.° Both contexts serve students from 
predominantly low socioeconomic statuses, though they differ in geographic 

locales and racial demographics. CHS is in a large metropolitan city while the 

schools in Louisiana range in location from rural areas to large towns. The 

schools in Louisiana tended to be predominantly Black in more urban locales 

and majority white in more rural areas. CHS was more racially diverse with 
approximately 60 percent of students identifying as Hispanic, 20 percent as 

Black, 5 percent as Asian, 5 percent as white, and 5 percent as two or more 

races.® These two scenarios are intended to illustrate how opportunities for 

and constraints to creating sociocultural learning environments for students 

are complexly influenced by meso- and micro-level factors (in addition to 
macro-level factors). In this approach, we intend to “attend equally to the 

wider policy stage” where “teachers learn to teach and to the more fine- 
grained, context-bound, and complex processes of how they learn to teach 

their subject in particular school contexts” (Gatti, 2016, p. 33). As a result, 

these examples are not prescriptive or exhaustive, but demonstrate three 


practical principles of sociocultural rubric conceptualization: 


1. The specific tensions and constraints educators face in conceptual- 
izing and implementing sociocultural writing instruction can vary 


based on meso- and micro-level contextual factors. 


2. These context-specific tensions and constraints present different 
opportunities for buffering, resisting, and/or reappropriating state 


writing rubrics, assessments, and curricula.’ 
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3. Sociocultural conceptualization and implementation must account 
for these constraints and opportunities in crafting more coherent 
socioculturally informed writing instruction. 


These principles are “practical” in the sense that they speak to the complex 
sociocultural and political conditions that practitioners work within, but 
not in the sense that they are easy to implement or translate to a particular 
context. 


Rubric Conceptualization in Context 


We view scenario one, the Louisiana context, as generally emphasizing as- 
sessment for accountability while scenario two, Consortium High School, is 
explicitly working toward assessment for pedagogy. Neither context is tension- 
free, but the level and types of constraints teachers face are quite different, 
as shown in Figure 3. In Louisiana, teachers face pressures to teach to the 
end-of-year ELA exam since the mandated state curricula are aligned with 
the test and because student performance on the state test is used to evaluate 
both school and teacher “quality.” These policies promote assessment for ac- 
countability, scientific management, and implicitly equate performance on 
DWAs as learning, all of which are in contention with sociocultural learning 
theories. This tension manifests in many teachers’ frustration at their limited 
input in the curriculum and in the limited types of texts and writing that are 


of constraints & pedagogy focus focus 


Lower levels | Sociocultural Accountability | Higher levels 
of constraints 


——<—<—<$—— 9 —/o 


Consortium High School 


* State tests not emphasized most of the year 
* Supports provided after low state test 
performance 


* Trust in relationships among administrators, 


teachers, students, and parents 

*Strong collaboration routines among 
administrators and teachers 

* Committed to instruction that is socially, 
culturally, and politically responsive 

* Primarily use rubrics developed by a 
collective of teachers except during 
“test-prep” 


ELA in Louisiana 


* State tests are primary focus of curricula 

* Punitive repercussions for teachers, schools, 
and students for low state test performance 

* Top-down surveillance and compliance 
predominant in relationships between 
administrators, teachers, and students 
*Weak or no collaboration routines among 
administrators and teachers 

* Committed to instruction that increases 
student achievement on state tests 

* Use mandated state rubrics that are used to 
assess state exam. 


Figure 3. Spectrum of Some Micro- and Meso-level Constraints to Implementing Sociocultural 


Writing Instruction 
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taught due to the focus on the state test.’ At CHS, the stakes of the state ELA 
exam are also high—it is a high school graduation requirement—but teachers 
and administrators in this school do not see teaching to the exam as their 
primary goal. Instead, for the majority of the year ELA teachers engage in 
reading and writing tasks that are related to current events and local issues 
that are important to students. Closer to the exam date, teachers explicitly 
engage ina “test-prep” curriculum that more closely aligns with behaviorist 
learning theory as well as the content and format of the state test. 

To understand how rubrics could be leveraged in these contexts, we 
need to understand different ways conceptualization and implementation 
of rubrics can unfold. In Louisiana, teachers report high levels of curricular 
control. When we asked teachers in interviews the extent to which the state 
test influenced their decisions about writing instruction, common answers 
included, “Pretty much 100%,” “That’s the sole influencer,” and “Oh, ev- 
erything. We technically teach to the test. We like to pretend that we don’t, 
but we do.” In the interviews, most teachers perceived that they must follow 
the curriculum “to the letter,” though there were exceptions. A few teachers 
talked about “sneaking in” other materials with their colleagues, “not being 
afraid to branch out into the real world,” and just aligning their instruction 
with the state rubric because its focus on claim, evidence, and explanation 
is “basically good writing.” Part of the mandated state curriculum includes 
using the state rubrics to assess student writing. In a survey we conducted 
in September 2018, all but one of the 44 teachers claimed that they usually 
use rubrics related to or adapted from the state exam and 93 percent (41) re- 
ported regularly using these rubrics to assess their students’ writing. Eighty- 
six percent (38) of teachers agreed with the statement that they would use 
a rubric to assess student writing even if it wasn’t required. In addition, 95 
percent (42) agreed that rubrics help students focus on conceptual features 
of their writing. While the state writing rubrics are a mandated policy tool 
attached to state ELA tests, these numbers suggest teachers may also see 
rubrics as a useful assessment and/or pedagogical tool. 

In terms of constraints, the state rubric is limiting in that it only focuses 
on two genres, literary analysis and research simulation.’ In addition, most 
teachers reported that the texts and essay prompts in their curricula were 
mandated. From our review of classroom tasks the teachers submitted to 
us, it seems the assigned texts were on or above grade level, but we believe 
they lacked relevant and/or controversial topics that students would find 
socially, culturally, or politically engaging. Teachers reported mixed feel- 
ings around mandated tasks. They typically viewed the texts as sufficiently 
complex and rigorous, though they sometimes lamented that a few of the 
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texts were “boring” and “not engaging” for students. 

Despite the numerous constraints in the Louisiana scenario, we still 
see some opportunities for sociocultural instruction. For one, the state rubric 
focuses on claim, evidence, and explanations, which are substantive aspects 
of student writing."° Given that (a) the rubric focuses on complex and substan- 
tive writing concepts, and (b) the specific ways in which teachers use the 
rubric is not easily observed or controlled by their supervisors, we believe 
this rubric could support (some) sociocultural rubric conceptualization and 
implementation. However, this would require teachers to reappropriate the 
meaning and function of the rubric. Instead of the rubric functioning pri- 
marily as a way to give students a grade and explain what will be expected 
of them on the state test (i.e., assessment for accountability), the rubric has 
potential to be reconceptualized and used as a pedagogical tool for develop- 
ing students’ writing skills and understandings. 

When engaging in text discussions and writing instruction, the con- 
cepts within the rubric can act as an anchor for students’ making meaning 
within a particular writing genre and its associated skills. For instance, one 
key concept in argument writing is evidence use. Teachers and students 
could examine how the concept of what qualifies as “good evidence use” 
varies between different types and genres of texts. In this case and others, 
iterative student-teacher discussion across classes, texts, and units about 
what constitutes “good” and “effective” writing within and across genres 
could better support the development of students’ understandings and skills 
in core writing concepts. In addition to aligning with sociocultural learning 
theories and promoting a dialogic learning environment, this iterative use 
of rubrics would also address a practical concern that 75 percent (33) of 
Louisiana teachers in our sample expressed on our survey: students struggle 
to interpret rubrics. 

Consortium High School (CHS) is part of a collective of 38 public 
high schools that have a waiver from New York State to replace four out of 
the five state Regents exams required for graduation with teacher-created 
performance-based assessment tasks (PBATs).''! A PBAT has two components: 
a domain-specific argument essay and an oral presentation of the essay in 
front of a panel of teachers. While teachers at each school design their own 
PBAT tasks, the rubrics used to assess PBATs are common and collaboratively 
developed by teachers across the collective.'? At CHS, students typically cre- 
ate and defend their PBATs in 11th and 12th grade. To prepare, students at 
CHS engage in similar tasks throughout 9th and 10th grade. The common 
rubrics and adaptations of it are used throughout the school to assess student 
writing. The PBAT model of instruction and assessment is highly valued by 
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all CHS staff, not only because it replaces Regents exams but also because it 
is viewed as a more authentic and meaningful way for students to develop 
and share their learning. At CHS (and generally across the collective), teach- 
ers seek to develop curricula and PBATs that engage students in discussion 
and action around controversial topics that are relevant to students’ lives. 

However, students in 10th grade must still pass the ELA Regents as a 
graduation requirement. As a result, ELA teachers who prepare students at 
CHS for the ELA exam have to negotiate the tensions between the sociocul- 
tural model of the PBATs and behaviorist assumptions implicit in New York’s 
test-based accountability system. To do so, ELA teachers explicitly engaged 
in a “double curriculum” to balance the competing demands and values of 
the PBAT curriculum and the state exam. For the majority of the year, the 
ELA teachers implemented a reading-writing workshop, where students 
spent extended periods of time reading and collaborating on writing with 
each other. Teachers worked together in a professional learning community 
(PLC) to develop curricula, assessments, and to reflect on student work. 
Their teacher-created units explored different genres of writing (e.g., poetry, 
memoir, dystopian fiction) and/or particular topics (e.g., human rights, 
gentrification) that were selected to be socially, culturally, and politically 
relevant to students. Classroom instruction aimed to promote the develop- 
ment of dialogic spaces, where students and teachers co-construct meaning 
around texts and related current events during class discussions. Writing 
workshops, which consisted of students working in small groups, focused 
on helping students develop their ideas and voice through writing. While 
these types of curricula and instruction align with sociocultural theory and 
the Consortium model of PBATs, they were in contention with the demands 
of the ELA Regents. 

The tensions and constraints ELA teachers face intensified as the 
exam dates approached.” About a month prior to the Regents, ELA teachers 
at CHS explicitly and intentionally shifted instruction to “test-prep.” Test- 
prep included direct instruction about the test format, the state rubrics, 
and test-taking strategies. Students then practiced these testing strategies 
on prior years’ tests and teachers provided strategic feedback to focus stu- 
dents on improving specific skills that were known to be emphasized on the 
test. Similar to the first scenario, the state ELA exam included two writing 
genres: a nonfiction argument essay and a literary analysis essay. This was 
a stressful and dreaded time of year for both teachers and students, despite 
the fact that the large majority of students typically passed the exam. The 
combination of the PBAT model of instruction and the department-level 
PLCs at CHS also created unique tensions between the ELA teachers and 
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other teachers. Because the only Regents exam administered to students was 
in ELA, this small group of teachers were often de facto tasked with these 
test-prep responsibilities. It was usually only in their classes that test-prep 
occurred. In addition, the ELA teachers would run sessions and mock exams 
after school, which also required them to score student essays. Over time, 
these teachers became more vocal at meetings about how they viewed these 
responsibilities as an unshared burden. 

While both CHS and Louisiana teachers dealt with the macro-level 
pressures of state accountability testing, the meso- and micro-level con- 
straints were quite different. Unlike teachers in Louisiana who usually 
had mandated ELA curricula, teachers in CHS were encouraged to develop 
authentic and relevant units rather than an accountability-driven curricu- 
lum and assessments. This gave CHS teachers more opportunities to (re) 
conceptualize how writing rubrics could support their pedagogy and student 
learning. Instead of providing measures for test-based accountability, the 
design, conceptualization, and implementation processes of Consortium 
rubrics were grounded in teacher collaboration and inquiry. This aligned 
with a similarly collaborative approach in the classroom, where the rubrics 
supported co-construction of knowledge with students around writing and 
textual analysis. While both groups had access to writing rubrics that could 
anchor classroom discussions of texts and writing, the state rubrics were 
often conceptualized as accountability tools (i.e., primarily a summative 
assessment of student achievement). However, we believe these state rubrics 
have some potential to be reconceptualized and reappropriated as peda- 
gogical tools. In both contexts, if teachers use rubrics to iteratively develop 
meaning around writing concepts and its authentic purposes, then rubrics 
can support the creation of sociocultural spaces. Still, the supportive and 
socioculturally aligned micro- and meso-features of CHS make it easier for 
teachers in that environment to both conceptualize and implement rubrics 
as pedagogical tools. 


Conclusion: Rubric Conceptualization as Resistance 


We want to be clear that our approach to rubric conceptualization assumes 
the current, less-than-ideal U.S. educational system where a range of histori- 
cal inequalities, structural barriers, and ideological tensions exist. Given the 
long history of test-based accountability in the United States that is rooted in 
behaviorism (Shepard, 2000), we believe this system is unlikely to change 
in the short term. As a result, we assume that administrators, teachers, 
and teacher educators likely work within a system that presents significant 
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macro-level constraints and obstacles to sociocultural conceptualization, but 
where there are also (some) opportunities to exert agency at the classroom 
(nicro) and school (meso) levels. 

Given these obstacles, we view engagement in sociocultural rubric 
conceptualization and implementation—whatever its form and depth—as 
not just a pedagogical decision but a form of principled resistance, which 
involves “overt or covert acts that reject instructional policies, programs, or 
other efforts to control teachers’ work that undermine or contradict profes- 
sional principles” (Achinstein & Ogawa, 2006, p. 32). Principled resistance 

also seeks to build alliances among educators, 
We view engagementin socio- students, parents, and communities (Anderson 
cultural rubric conceptualization —& Cohen, 2015). In the two scenarios discussed 
and implementation—whatever here, the states’ test-based accountability sys- 
its form and depth—as not justa tems are a policy tool that controls pedagogical 
pedagogical decision butaform and discursive aspects of the teachers’ work. Au 
of principled resistance. (2009) and other critical scholars have called this 
phenomenon “steerage at a distance”: 


[H]igh-stakes testing is having a tangible impact on the educational ex- 
periences of students. Thus it is important to recognize that high-stakes 
testing is in fact changing the educational environments of schools: Not 
only do teachers lose control of curricular decisions, but any power the 
students might have as contributors to their own educational process is 
also taken away. (p. 101) 


While the macro-level pressures of the test-based accountability systems in 
our two scenarios did not entirely determine the classroom instruction in 
our two scenarios, they certainly influenced it. 

We believe there are two key ways that teachers’ use of sociocultural 
conceptualizations of rubrics specifically, and writing instruction generally, 
allow them to use their agency as professionals to resist the problematic as- 
sumptions and consequences of test-based accountability systems. The first 
is that sociocultural conceptualizations have the potential for teachers and 
students to position themselves as (co-)creators and inquirers of knowledge. 
Second, and relatedly, sociocultural conceptualizations have the potential to 
resist the systemic disempowerment of students and teachers. 

In Louisiana, teachers were positioned as transmitters of knowledge 
by the state and districts because their role was confined to the delivery of 
mandated curricula. For teachers to use their professional judgment to adapt 
or change the mandated curriculum was a sanctionable offense. Similarly, in 
learning to write text-based arguments, students (like teachers) were restrict- 
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ed to engaging with the writing forms, topics, and texts deemed relevant to 
the test. As a result, students were effectively recapitulating the perspectives 
of the texts’ authors, though under the pretext that they were formulating 
their “own” arguments. We believe that the tight alignment between the 
state, district, and school mandates, along with the punitive assessment for 
accountability function, greatly constrained teachers’ agency and ability 
to conceptualize and implement more sociocultural forms of teaching and 
learning. This is not to say that teachers could not resist or that they had no 
responsibility to resist. However, teacher compliance and deprofessionaliza- 
tion are also likely symptoms of the systemic and institutional failings of 
test-based accountability systems. Sociocultural rubric conceptualization 
is one strategy that we believe could help teachers imagine and implement 
alternative pedagogies that resist the behaviorist content, pedagogy, and 
implied aims incentivized by mandated curricula and assessments. 

In contrast, the meso- and micro-contexts at CHS created more op- 
portunities to resist the transmission model of teaching and learning. ELA 
teachers took advantage of these opportunities by designing, conceptualizing, 
and implementing writing curricula that was responsive to student interests 
and the current and historical events affecting their lives. CHS’s commitment 
to sociocultural models of instruction and the common rubrics for PBATs 
supported the ELA teachers in both imagining and implementing a class- 
room environment that situated students and themselves as co-creators. This 
environment thus supported student and teacher empowerment by situating 
both groups as inquirers and (co-)constructors of knowledge that is informed 
by and relevant to their lives. While these sociocultural conceptualizations/ 
resistance did not (and could not) fully resolve the ideological tensions that 
existed in the teachers’ commitment to sociocultural instruction and the 
demands of the ELA Regents, it did provide ways for teachers to reposition 
themselves and their students as active epistemic agents. 

Within this relative agency at CHS, teachers and students were able 
to work collectively and individually to construct understandings of the 
world through reading and writing, rather than merely transmit and com- 
ply with top-down educational policies and mandates. While the resistance 
of individuals cannot resolve the historical tensions between behaviorist 
and sociocultural learning theories, let alone the structural injustices that 
test-based accountability systems perpetuate, it can lay the groundwork for 
more coordinated activism. In this resistance, teachers, administrators, and 
community members can work together to construct learning environments 
that are dialogic and responsive to local needs and values. 
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Notes 


1. This is also not to say that rubric design and state tests do not impose real, and 
sometimes insurmountable, constraints to writing instruction. We discuss a scenario 
with many design-based constraints in “What Could Sociocultural Rubric Conceptu- 
alization Look Like?” later in this article. 

2. The role(s) of administrators can vary within the design of writing rubrics. 
They may work with policymakers or curriculum designers to create rubrics, and/ 
or they may work with teachers at the school level to interpret and adapt rubrics for 
instruction. 

3. These shortcomings of DWAs can be described as having low consequential 
validity, which is “an appraisal of the potential social consequences of the proposed 
use and of the actual consequences when used” (Messick, 1980, p. 1023). Behizadeh 
(2014) has argued that DWAs have limited consequential and construct validity due to 
their lack of communicative function, sociocultural relevance, and authentic purpose. 

4. Consortium High School is a pseudonym. 

5. emily was certified to teach special education at CHS. She co-taught with other 
teachers across content areas including history, literature, science (biology, physical 
science), and math (algebra). However, she never taught with the ELA teachers at CHS. 
As a result, the scenario she describes is based on her observations and conversations 
with ELA teachers and their students. 

6. The descriptions of locales and racial demographics for both scenarios are based 
on data from the National Center for Education Statistics (NCES). 

7. For more on the concepts of bridging and buffering, see Honig and Hatch (2004). 
We view buffering, or “strategically deciding to engage external demands in limited 
ways” (p. 23), as relevant to sociocultural rubric conceptualization since it is often 
incompatible with external demands of state based accountability systems. 

8. However, it is worth noting that some teachers, who view the mandated cur- 
riculum’s focus on state test-based skills as useful and aligned with their goals for 
their students, do not experience this tension. This further emphasizes our point that 
tensions are a result of being oriented toward sociocultural learning theories and 
pedagogical functions of rubrics and assessment. 

9. Research simulation writing tasks are based on multiple nonfiction texts that 
ask students to synthesize and analyze information across texts. 

10. Teachers repeatedly explained that grammar and mechanics are not empha- 
sized on the state exam and that, as a result, the curriculum and the rubric do not 
emphasize them either. 

11. Students must present a PBAT in math, science, and history to mirror the 
Regents exam requirements. The fourth PBAT can be in literature or a second PBAT 
in any of the other subjects. 

12. There are annual meetings to help teachers and schools “norm” their PBATs 
in relation to the common rubrics. Teacher representatives from the Consortium 
schools bring their PBATs and corresponding student work. Tasks and student work 
are evaluated on how well they align with the Consortium rubrics. Similar teacher 
meetings also periodically review and revise each Consortium rubric. 

15. Regents exams are administered twice a year, in January and June. At CHS, 
the main test-prep period was for the June administration. 


References 


Achinstein, B., & Ogawa, R. T. (2006). (In)fidelity: What the resistance of new teach- 
ers reveals about professional principles and prescriptive education policies. 
Harvard Educational Review, 76(1), 30-63. 


356 


howe and Correnti >» Political and Pedagogical Tensions of Rubrics 


Anderson, G., & Cohen, M. I. (2015). Redesigning the identities of teachers and 
leaders: A framework for studying new professionalism and educator resis- 
tance. Education Policy Analysis Archives, 23(85), 1-29. doi.org/10.14507/ 
epaa.v23.2086 


Andrade, H. G., & Boulay, B. A. (2003). Role of rubric-referenced self-assessment 
in learning to write. The Journal of Educational Research, 97(1), 21-34. 
doi.org/10.2307/27542460 


Andrade, H. L. (2006). The trouble with a narrow view of rubrics. English Journal, 
95(6), 9. 


Andrade, H. L., Du, Y., & Wang, X. (2008). Putting rubrics to the test: The effect of a 
model, criteria generation, and rubric- referenced self-assessment on elemen- 
tary school students’ writing. Educational Measurement: Issues and Practice, 
27(2), 3-15. doi.org/10.1111/j.1745-3992.2008.00118.x 


Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthe- 
sis. Educational Researcher, 36(5), 258-267. doi.org/10.3102/0013189X07306523 


Au, W. (2009). Unequal by design: High-stakes testing and the standardization of 
inequality. Routledge. 


Au, W. (2011). Teaching under the new Taylorism: High-stakes testing and the 
standardization of the 21st century curriculum. Journal of Curriculum Studies, 
43(1), 25-45. doi.org/10.1080/00220272.2010.521261 


Au, W. (2016). Meritocracy 2.0: High-stakes, standardized testing as a racial project 
of neoliberal multiculturalism. Educational Policy, 30(1), 39-62. https:// 
doi.org/10.1177/0895904815614.916 


Baird, J. A., Andrich, D., Hopfenbeck, T. N., & Stobart, G. (2017). Assessment and 
learning: Fields apart? Assessment in Education: Principles, Policy & Practice, 
24(3), 317-350. doi.org/10.1080/0969594X.2017. 1319337 


Barnes, N., Fives, H., & Dacey, C. M. (2017). U.S. teachers’ conceptions of the 
purposes of assessment. Teaching and Teacher Education, 65, 107-116. 
doi.org/10.1016/j.tate.2017.02.017 


Behizadeh, N. (2014). Mitigating the dangers of a single story: Creating large-scale 
writing assessments aligned with sociocultural theory. Educational Researcher, 
43(3), 125-136. doi.org/10.3102/0015189X 14529604. 


Behizadeh, N., & Pang, M. E. (2016). Awaiting a new wave: The status of state writ- 
ing assessment in the United States. Assessing Writing, 25(41), 25-41. 


Ben-Porath, S. (2013). Deferring virtue: The new management of students and the 
civic role of schools. Theory and Research in Education, 11(2), 111-128. 
doi.org/10.1177/1477878513485172 


Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in 
Education: Principles, Policy & Practice, 18(1), 5-25. doi.org/10.1080/096959 
4X.2010.513678 


Broad, B. (2003). What we really value: Beyond rubrics in teaching and assessing 
wriling. Utah State University Press. 


Brookhart, S. M., & Chen, F. (2015). The quality and effectiveness of descriptive 
rubrics. Educational Review, 67(3), 343-368. doi.org/10.1080/00131911.2014.92 
9565 


357 


English Education, V52 N4, July 2020 


Correnti, R., Matsumura, L. C., Hamilton, L., & Wang, E. (2013). Assessing students’ 
skills at writing analytically in response to texts. The Elementary School Journal, 
114(2), 142-177. 


Gatti, L. (2016). Toward a framework of resources for learning to teach: Rethinking 
US teacher preparation. Springer Nature. 


Greeno, J. G., Collins, A. M., & Resnick, L. B. (1997). Cognition and learning. In 
D. C. Berliner & R. C. Caltee (Eds.), Handbook of educational psychology (pp. 
15-46). Simon & Schuster Macmillan. https://doi.org/10.1007/s13398-014-0173- 
7.2 


Gresalfi, M. S. (2009). Taking up opportunities to learn: Constructing dispositions 
in mathematics classrooms. Journal of the Learning Sciences, 18(3), 327-369. 
https://doi.org/10.1080/105084.00903013470 


Hamilton, L. (2003). Assessment as a policy tool. Review of Educational Research, 
27, 25-68. 


Harris, L. R., & Brown, G. T. L. (2009). The complexity of teachers’ conceptions of 
assessment: Tensions between the needs of schools and students. Assessment in 
Education: Principles, Policy & Practice, 16(3), 365-381. doi.org/10.1080/0969 
5940903319745 


Honig, M. I., & Hatch, T. C. (2004). Crafting coherence: How schools strategically 
manage multiple, external demands. Educational Researcher, 33(8), 16-30. 


Horsford, S. D. (2017). A race to the top from the bottom of the well? The paradox of 
race and U.S. education reform. The Educational Forum, 81, 136-147. 


Jennings, J. L., & Bearak, J. M. (2014). “Teaching to the Test” in the NCLB Era. Edu- 
cational Researcher, 43(8), 381-389. doi.org/10.3102/0013189x 14554449 


Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity 
and educational consequences. Educational Research Review, 2(2), 130-144. 
doi.org/10.1016/j.edurev.2007.05.002 


Kane, M., Crooks, T., & Cohen, A. (1999). Validating measures of performance. Edu- 
cational Measurement: Issues and Practice, 18(2), 5-17. 


Kinloch, V. (2010). Harlem on our minds: Place, race, and the literacies of urban 
youth. Teachers College Press. 


Kohn, A. (2006). The trouble with rubrics. English Journal, 95(4), 12-15. doi.org/ 
10.1192/pb.33.10.396a 


Ladson-Billings, G. (2014). Culturally relevant pedagogy 2.0: a.k.a the remix. Har- 
vard Educational Review, 84(1), 74-84. 


Lee, C. D. (2008). Cultural modeling as opportunity to learn. In P. A. Moss, D. 
Pullin, J. P. Gee, E. H. Haertel, & L. Jones Young (Eds.), Assessment, equily, and 
opportunity to learn (pp. 136-167). Cambridge University Press. doi.org/10.1017/ 
CBO9780511802157 


Mabry, L. (1999). Writing to the rubric: Lingering effects of traditional standard- 
ized testing on direct writing assessment. Phi Delia Kappan, 80(9), 673-674. 


Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 
35(11), 1012-1027. 


358 


howe and Correnti >» Political and Pedagogical Tensions of Rubrics 


Newell, G. E., Beach, R., Smith, J., & VanDerHeide, J. (2011). Teaching and learning 
argumentative reading and writing: A review of research. Reading Research 
Quarterly, 46(3), 273-304. doi.org/10/1598/RRQ.46.3.4 


Paris, D. (2012). Culturally sustaining pedagogy: A needed change in stance, term- 
inology, and practice. Educational Researcher, 41(3), 93-97. doi.org/10.3102/ 
0013189X124.41244 


Remesal, A. (2011). Primary and secondary teachers’ conceptions of assessment: 
A qualitative study. Teaching and Teacher Education, 27(2), 472-482. 
doi.org/10.1016/j.tate.2010.09.017 


Reznitskaya, A., Kuo, L. J., Glina, M., & Anderson, R. C. (2009). Measuring argumen- 
tative reasoning: What’s behind the numbers? Learning and Individual Differ- 
ences, 19(2), 219-224. doi.org/10.1016/j.lindif.2008.11.001 


Saddler, B., & Andrade, H. (2004). The writing rubric. Educational Leadership, 
62(2), 48-52. 


Shepard, L. A. (2000). The role of assessment in a learning culture. Educational 
Researcher, 29(7), 4-14. 


Shepard, L. A. (2013). Validity for what purpose? Teachers College Record, 115(9), 
1-12. 


Shepard, L. A., Penuel, W. R., & Pellegrino, J. W. (2018). Using learning and motiva- 
tion theories to coherently link formative assessment, grading practices, and 
large-scale assessment. Educational Measurement: Issues and Practice, 37(1), 
21-34. doi.org/10.1111/emip.12189 


Smit, R., & Birri, T. (2014). Assuring the quality of standards-oriented classroom as- 
sessment with rubrics for complex competencies. Siudies in Educational Evalua- 
tion, 43, 5-13. doi.org/10.1016/j.stueduc.2014.02.002 


Star, S. L., & Griesemer, J. R. (1989). Institutional ecology, “translations” and 
boundary objects: Amateurs and professionals in Berkeley’s museum of verte- 
brate zoology, 1907-1939. Social Studies of Science, 19, 387-420. 


Tuck, E. (2012). Urban youth and school pushout: Gateway, gel-aways, and the GED. 
Routledge. 


Tuck, E., & Gorlewski, J. (2016). Racist ordering, settler colonialism, and edTPA: 
A participatory policy analysis. Educational Policy, 30(1), 197-217. doi.org/ 
10.1177/0895904815616483 


Turley, E. D., & Gallagher, C. W. (2008). On the “uses” of rubrics: Reframing the 
great rubric debate. English Journal, 97(4), 87-92. 


Welner, K. G. (2013). Consequential validity and the transformation of rests from 
measurement tools to policy tools. Teachers College Record, 115(9), 1-6. doi. 
org/10.1002/jhet.5570060305 


Wilson, M. (2007). Why I won’t be using rubrics to respond to students’ writing. 
English Journal, 96(4), 62-66. 


359 


Reproduced with permission of copyright owner. Further 
reproduction prohibited without permission. 


